Multi-head Self-attention Mechanism Research Articles

Incorporating temporal multi-head self-attention convolutional networks and LightGBM for indoor air quality prediction

Friction between subway wheels and tracks and inadequate combustion of fuel are causes of respirable particulate matter. Restricted underground ventilation and high population density make it difficult for particulate matter to dissipate, posing a threat to human health. An effective data-driven model for indoor pollutant prediction can enhance preparedness for high pollution situations. This study introduces a feature extraction method that combines kernel principal component analysis with max-relevance and min-redundancy algorithm. The temporal convolutional network, enhanced with a multi-head self-attention mechanism, adeptly captures time series features and effectively manages attention weight allocation. Additionally, the incorporation of the light gradient boosting machine method enhances overall efficiency. The proposed framework for PM2.5 concentration prediction was employed to a high-traffic subway station in Seoul. In the test set, the model demonstrated strong performance with evaluation metrics including an R2 value of 0.92, RMSE of 6.02 μg/m3, MAE of 4.36, and MAPE of 20.58 μg/m3. Compared to the conventional LSTM, the proposed method reduces the RMSE by 20.5% and the MAPE by 49.05%. Notably, the model we propose demonstrates superior capabilities in managing large datasets and offers enhanced predictive accuracy compared to baseline models. It effectively addresses the limitations observed in models like LSTM, which often struggle with adequately capturing feature information, and overcomes the generalization weaknesses inherent in models such as the Transformer. This advancement significantly boosts the efficiency of environmental monitoring and fosters both automation and intelligence in the analysis of environmental data.

GraphPath: a graph attention model for molecular stratification with interpretability based on the pathway-pathway interaction network.

Studying the molecular heterogeneity of cancer is essential for achieving personalized therapy. At the same time, understanding the biological processes that drive cancer development can lead to the identification of valuable therapeutic targets. Therefore, achieving accurate and interpretable clinical predictions requires paramount attention to thoroughly characterizing patients at both the molecular and biological pathway levels. Here, we present GraphPath, a biological knowledge-driven graph neural network with multi-head self-attention mechanism that implements the pathway-pathway interaction network. We train GraphPath to classify the cancer status of patients with prostate cancer based on their multi-omics profiling. Experiment results show that our method outperforms P-NET and other baseline methods. Besides, two external cohorts are used to validate that the model can be generalized to unseen samples with adequate predictive performance. We reduce the dimensionality of latent pathway embeddings and visualize corresponding classes to further demonstrate the optimal performance of the model. Additionally, since GraphPath's predictions are interpretable, we identify target cancer-associated pathways that significantly contribute to the model's predictions. Such a robust and interpretable model has the potential to greatly enhance our understanding of cancer's biological mechanisms and accelerate the development of targeted therapies. https://github.com/amazingma/GraphPath. Supplementary data are available at Bioinformatics online.

Lightweight human activity recognition method based on the MobileHARC model

In recent years, Human activity recognition (HAR) based on wearable devices has been widely applied in health applications and other fields. Currently, most HAR models are based on the Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), or their combination. Recently, there have been proposals based on Transformer and its variant models. However, due to the fact that these models have sequential network structures and are unable to simultaneously focus on local and global features, thus, resulting in a reduction in recognition performance. In addition, along with the substantial computational resources required by Transformers, they are not suitable for resource-constrained devices. In this paper, the primary distinction of our proposed model from other hybrid models that combine CNN and Transformer is that our model adopts a completely new parallel network architecture and primarily focuses on lightweight design. Particularly, We proposed the Mobile Human Activity Recognition Conformer (MobileHARC), which adopts the parallel structure with a lightweight Transformer and CNN as the backbone networks. Furthermore, we proposed the Inverted Residual Lightweight Convolution Block and Multiscale Lightweight Multi-Head Self-Attention Mechanism. We systematically evaluate the proposed models on four public datasets. Experimental results show that MobileHARC achieves superior recognition performance, and uses fewer Floating-Point Operations per Second (FLOPs) and parameters compared to current models.

CourseKG: An Educational Knowledge Graph Based on Course Information for Precision Teaching

With the rapid development of advanced technologies, such as artificial intelligence and deep learning, educational informatization has entered a new era. However, the explosion of information has brought numerous challenges. Knowledge graphs, as a crucial component of artificial intelligence, can contribute to the quality of teaching. This study proposes an educational knowledge graph based on course information named CourseKG for precision teaching. Precision teaching seeks to individualize the curriculum for each learner and optimize learning efficiency. CourseKG aims to establish a correct and comprehensive curriculum knowledge system and promote personalized learning paths. CourseKG can address the issue that current general-purpose knowledge graphs are not suitable for the education field. Particularly, this study proposes a framework for educational entity recognition based on the pre-trained BERT model. This framework captures relevant information in the educational domain using the BERT model and combines it with the BiGRU and multi-head self-attention mechanism to extract multi-scale and multi-level global dependency relationships. In addition, the CRF is used for character-label decoding. Further, a relationship extraction method based on the BERT model, which integrates sentence features and educational entities and estimates the similarity between knowledge pairs using cosine similarity, is proposed. The proposed CourseKG is verified by experiments using real-world C programming course data. The experimental results demonstrate the effectiveness of CourseKG. Finally, the results show that the proposed CourseKG can significantly enhance the precision teaching quality and realize multi-directional adaptation among teachers, courses, and students.

A recursive multi-head self-attention learning for acoustic-based gear fault diagnosis in real-industrial noise condition

Acoustic-based diagnosis (ABD) is a promising method for rotating machinery fault detection in real-industrial fields due to its advantage of non-contact measurement by air-couple. However, most ABD methods, which rely on single function module of denoising or anti-noise diagnosis, lacks sufficient ability against the strong and highly non-stationary background noise interference in practical industrial application. To deal with the obstacle, a novel two-stage ABD system, which performs denoising and anti-noise diagnosis in a step-by-step fashion, is proposed in this paper. In our method, the denoising sub-system can adaptively track and suppress real-industrial background noise from collected multi-channel signal by our designed recursive multi-head self-attention (RMHSA) mechanism, which take into account the global and long-range information of signal to recursively exclude noise component at each position in signal sequence for overcoming the limitation of traditional method in balancing denoising performance and computational complexity and discontinuity in diagnosis procedure. Subsequently, a novel anti-noise method is explored to automatically track the residual noise components after initial suppression and estimating the noise interference probability within time-frequency (T-F) unit of each signal sample by our designed spatial multi-head self-attention (SMHSA) mechanism in recursive manner. Simultaneously, the anti-noise protocol interacts with diagnosis model based on estimated probability in anti-noise diagnosis sub-system to further improve the diagnosis performance of ABD system under the interference of background noise. Experiment result in both real-industrial noise condition and simulated noise conditions with extremely worse SNRs indicate that the proposed two-stage ABD system is effective in dealing with gear fault diagnosis task under noise condition.

A fast gangue detection algorithm based on multi-head self-attention mechanism and anchor frame optimization strategy

ABSTRACT Because of the problem of difficult identification, model acceleration, and localization for coal gangue detection, a fast coal gangue detection algorithm based on a multi-head self-attention mechanism and anchor frame optimization strategy is designed. Firstly, the multi-head self-attention mechanism MHSA is combined with the CNN backbone structure of YOLOv5s, which is used to enhance the algorithm’s ability to perceive global information to strengthen the model’s recognition of coal and gangue; Secondly, the lightweight Ghost module will be used to reconfigure the backbone of YOLOv5s and the feature fusion layer to reduce the complexity of the model and increase the speed of the coal gangue image detection; And lastly, the selection of an appropriate strategy of input image scale and anchor frame size to improve the performance of the detection algorithm to solve the problem of localization of coal and gangue. Experiments show that the average accuracy of the improved MYL model mAP_0.5 reaches 91.8%, the detection accuracy of mAP_0.5 is improved by 3.6%, and the detection speed is 80 FPS. The improved MYL model can show stronger robustness and superior performance in the detection of coal gangue and can meet the requirements of effective industrial detection.

PPTtrack: Pyramid pooling based Transformer backbone for visual tracking

In visual tracking, Convolutional Neural Network (CNN) is usually used as feature extractor, and can fully explore local dependencies of image blocks, which is help for improving tracking performance. However, CNN ignores global dependencies in image blocks. The global modeling is crucial in visual tracking. Recently, Transformer has gained attention to fully explore global dependencies on sequential data. However, Transformer’s unique multi-head self-attention mechanism results in high computational complexity. In this paper, we design a pyramid pooling based Transformer backbone network for visual tracking. Pyramid pooling refers to multiple pooling operations for feature map with different receptive fields and strides. The output data of each pooling layer is concatenated to form the final pooled feature map. On the one hand, after flattening the feature map with pyramid pooling, its sequence length will be greatly reduced. This will effectively reduce the computational complexity of the multi-head self-attention. On the other hand, pyramid pooling can extract multi-scale features, makes the feature maps contain more global context information. Finally, we propose a novel tracker with the designed pyramid pooling based Transformer backbone network and the Transformer based model predictor. We train the proposed tracker by end-to-end, and evaluate it on seven tracking benchmarks including UAV123, NFS, Trackingnet, LaSOT, GOT-10K, VOT2020 and RGBT2019. The proposed tracker achieves 79.8% robustness and 35 FPS on the VOT2020 dataset. The experiment demonstrates that proposed tracker achieves superior tracking performance with state-of-the-art trackers.

Clinical deep model to analyse medical multivariate time-series data for health diagnosis

ABSTRACT Clinical auxiliary decision-making is related to life and health of patients, so the deep model needs to extract the personalised representation of patients to ensure high analysis and prediction accuracy; and provide a basis for prediction conclusions. In this context, a clinical deep model proposed an interpretable assessment method of patient health status based on contextual learning of medical features, encoding the time-series features of each variable separately, and using a multi-head de-coordination self-attention mechanism for learning Relationships between different features; feature skip-connection encoding based on a compressed excitation mechanism is proposed to improve the sensitivity of the model.

Intelligent Detection Method for Satellite TT&C Signals under Restricted Conditions Based on TATR

In complex electromagnetic environments, satellite telemetry, tracking, and command (TT&C) signals often become submerged in background noise. Traditional TT&C signal detection algorithms suffer a significant performance degradation or can even be difficult to execute when phase information is absent. Currently, deep-learning-based detection algorithms often rely on expert-experience-driven post-processing steps, failing to achieve end-to-end signal detection. To address the aforementioned limitations of existing algorithms, we propose an intelligent satellite TT&C signal detection method based on triplet attention and Transformer (TATR). TATR introduces the residual triplet attention (ResTA) backbone network, which effectively combines spectral feature channels, frequency, and amplitude dimensions almost without introducing additional parameters. In signal detection, TATR employs a multi-head self-attention mechanism to effectively address the long-range dependency issue in spectral information. Moreover, the prediction-box-matching module based on the Hungarian algorithm eliminates the need for non-maximum suppression (NMS) post-processing steps, transforming the signal detection problem into a set prediction problem and enabling parallel output of the detection results. TATR combines the global attention capability of ResTA with the local self-attention capability of Transformer. Experimental results demonstrate that utilizing only the signal spectrum amplitude information, TATR achieves accurate detection of weak TT&C signals with signal-to-noise ratios (SNRs) of −15 dB and above (mAP@0.5 > 90%), with parameter estimation errors below 3%, which outperforms typical target detection methods.

A hybrid deep learning method for the prediction of ship time headway using automatic identification system data

Ship Time Headway (STH) is used in maritime navigation to describe the time interval between the arrivals of two consecutive ships in the same water area. This measurement may offer a straightforward way to gauge the frequency of ship traffic and the likelihood of congestion in a particular area. STH is an important factor in understanding and managing the dynamics of ship movements in busy waterways. This paper introduces a hybrid deep learning method for predicting STH in time domain. The method integrates the Seasonal-Trend Decomposition using Loess (STL), Multi-head Self-Attention (MSA) mechanism into Long Short-Term Memory (LSTM) neural network. The STH dataset was extracted from the Automatic Identification System (AIS) through ship trajectory spatial motion, and the seasonal, trend and residual components of the decomposition were then determined from the STH dataset using the STL algorithms. MSA-LSTM is adopted to comprehensively capture the evolving patterns of STH from the sequence. Comparison studies with existing methods demonstrate the accuracy and robustness of the predictions provided by this method, indicating that the proposed method outperforms other models in terms of prediction performance and learning capabilities. By predicting STH, the method offers potential to assist maritime traffic managers and navigators in assessing ship flow, thereby enabling them to make informed decisions on navigation safety and efficiency.

Combining permuted language model and adversarial training for Chinese machine reading comprehension

Due to the polysemy and complexity of the Chinese language, Chinese machine reading comprehension has always been a challenging task. To improve the semantic understanding and robustness of Chinese machine reading comprehension models, we propose a model that utilizes adversarial training algorithms and Permuted Language Model (PERT). Firstly, we employ the PERT pre-training model to embed paragraphs and questions into vector space to obtain corresponding sequential representations. Secondly, we use a multi-head self-attention mechanism to extract key textual information from the sequence and employ a Bi-GRU network to semantically fuse the output feature vectors, aiming to learn deep semantic representations in the text. Finally, we introduce perturbations into the model training process. We achieve this by utilizing adversarial training algorithms such as Fast Gradient Method (FGM) and Projected Gradient Descent (PGD). These algorithms generate adversarial samples to enhance the model’s robustness and stability when facing diverse inputs. We conducted comparative experiments on the publicly available Chinese reading comprehension datasets CMRC2018 and DRCD. The experimental results show that our proposed model has achieved significant improvements in both EM and F1-Score compared to the baseline model. To validate the model’s generalization and robustness, we utilized ChatGPT to construct a scientific dataset that includes a large number of domain-specific terms, sentences with mixed Chinese and English, and complex comprehension tasks. Our model also performed remarkably well on the self-built dataset. In conclusion, the proposed model not only effectively enhances the understanding of semantic information in Chinese text but also demonstrates a certain level of generalization capability.

Micro LED defect detection with self-attention mechanism-based neural network

We propose a method utilizing a YOLO detector for the precise localization of defective chips and the identification of defect types within multi-scale multi-target images. To address the challenge of optimizing training costs and enhancing model generalization, we introduce an end-to-end deep neural network, CM-YOLOv5, specifically designed for chip detection. We incorporate a novel bottleneck layer, MA-CSP, in conjunction with Multi-Head Self-Attention mechanism (MHSA). Additionally, we propose a class-balanced loss function (CB-BCE Loss) to tackle the issue of uneven distribution of defective samples in the Micro LED dataset. To further enhance convergence speed and detection precision, we introduce the SIoU Loss combined with Meta-AconC. Our experimental results, conducted on the Micro LED dataset, demonstrate notable improvements with CM-YOLOv5 over the basic YOLOv5 algorithm. Specifically, CM-YOLOv5 exhibits a 3.8 % increase in mean average precision and a 3.7 % improvement in precision, surpassing current mainstream object detection algorithms, including YOLOR, YOLOX, and YOLOv6, etc., in terms of general evaluation metrics. Finally, upon deploying our proposed algorithm on the edge device NVIDIA Jetson Xavier NX, CM-YOLOv5 demonstrates commendable speed and detection performance in embedded scenarios.

Local feature expansion Vision Transformer model for bearing fault diagnosis under noise environments

Vision Transformer (ViT) shows potential in bearing fault diagnosis due to its multi-head self-attention mechanism and parallel feature extraction network which are efficient to achieve the robust complete feature representation of the fault. However, its adaption to the noise interference relies on the sufficient huge amount of training samples to prepare the local features of the fault and may suffer performance degradation when only a limited number of samples are available for the model training. To combat this challenge, an improved ViT diagnosis model based on the local feature expansion, i.e., LFE-ViT, is proposed. An auxiliary feature extraction block is introduced using a local feature expansion network and works as a parallel module with the ViT encoder. Through the enlargement of the receptive field, the multi-scale local features on a high dimensional space are available upon the limited samples. Then, through a feature embedding channel, the extracted local features are transmitted to the ViT encoder. Finally, by virtue of the multi-head self-attention mechanism to capture the time sequence global information, a fault diagnosis model comprising comprehensively local and global feature information is derived. Experimental validation on the bearing fault dataset from Case Western Reserve University shows that LFE-ViT has provided a rather satisfactory diagnosis performance under limited samples and noise environment.

MRNDR: Multihead Attention-Based Recommendation Network for Drug Repurposing.

As is well-known, the process of developing new drugs is extremely expensive, whereas drug repurposing represents a promising approach to augment the efficiency of new drug development. While this method can indeed spare us from expensive drug toxicity and safety experiments, it still demands a substantial amount of time to carry out precise efficacy experiments for specific diseases, thereby consuming a significant quantity of resources. Therefore, if we can prescreen potential other indications for selected drugs, it could result in substantial cost savings. In light of this, this paper introduces a drug repurposing recommendation model called MRNDR, which stands for Multi-head attention-based Recommendation Network for Drug Repurposing. This model serves as a prediction tool for drug-disease relationships, leveraging the multihead self-attention mechanism that demonstrates robust generalization capabilities. These capabilities stem not only from our extensive million-level training data set, BioRE (Biology Recommended Entity data), but also from the utilization of the WRDS (Weighted Representation Distance Score) algorithm proposed by us. The MRNDR model has achieved new state-of-the-art results on the GP-KG public data set, with an MRR (Mean Reciprocal Rank) score of 0.308 and a Hits@10 score of 0.628. This represents significant improvements of 4.7% (MRR) and 18.1% (Hits@10) over the current best-performing models. Additionally, to further validate the practical utility of the model, we examined results recommended by MRNDR that were not present in the training data set. Some of these recommendations have undergone clinical trials, as evidenced by their presence on ClinicalTrials.gov and the China Clinical Trials Center, indirectly confirming the applicability of MRNDR. The MRNDR model can predict the reusability of candidate drugs, reducing the need for manual expert assessments and enabling efficient drug repurposing.

Res-BiANet: A Hybrid Deep Learning Model for Arrhythmia Detection Based on PPG Signal

Arrhythmias are among the most prevalent cardiac conditions and frequently serve as a direct cause of sudden cardiac death. Hence, the automated detection of arrhythmias holds significant importance for assisting in the diagnosis of heart conditions. Recently, the photoplethysmography (PPG) signal, capable of conveying heartbeat information, has found application in the field of arrhythmia detection research. This work proposes a hybrid deep learning model, Res-BiANet, designed for the detection and classification of multiple types of arrhythmias. The improved ResNet and BiLSTM models are connected in parallel, and spatial and temporal features of PPG signals are extracted using ResNet and BiLSTM, respectively. Subsequent to BiLSTM, a multi-head self-attention mechanism was incorporated to enhance the extraction of global temporal correlation features over long distances. The model classifies five types of arrhythmia rhythms (premature ventricular contractions, premature atrial contractions, ventricular tachycardia, supraventricular tachycardia, and atrial fibrillation) and normal rhythm (sinus rhythm). Based on this foundation, experiments were conducted utilizing publicly accessible datasets, encompassing a total of 46,827 PPG signal fragments from 91 patients with arrhythmias. The experimental results demonstrate that Res-BiANet achieved exceptional classification performance, including an F1 score of 86.88%, overall accuracy of 92.38%, and precision, sensitivity, and specificity of 88.46%, 85.15%, and 98.43%, respectively. The outstanding performance of the Res-BiANet model suggests significant potential in supporting the auxiliary diagnosis of multiple types of arrhythmias.

Abstractive text summarization model combining a hierarchical attention mechanism and multiobjective reinforcement learning

Text summarization research is significant and challenging in the domain of natural language processing. Abstractive text summarization mainly uses the encoder-decoder framework, wherein the encoder component does not have a sufficient semantic comprehension of the input text, and there are exposure biases and semantic inconsistencies between the reference and generated summaries during the training process. We propose an improved encoder-decoder model that incorporates a hierarchical attention mechanism and multiobjective reinforcement learning. The encoder introduces a multihead self-attention mechanism to allow for the acquisition of more comprehensive semantic information from multiple angles and dimensions, while the decoder introduces a pointer-generator network to solve the out-of-vocabulary problem. Multiobjective reinforcement learning methods are constructed throughout the training process to optimize the model in terms of addressing exposure bias, maintaining semantic consistency, and enhancing readability. The results of the comparative experiments demonstrate that the proposed model significantly improved in terms of the ROUGE evaluation metric, and the generated summaries were semantically similar to the reference summaries.

YOLO-Rlepose: Improved YOLO Based on Swin Transformer and Rle-Oks Loss for Multi-Person Pose Estimation

In recent years, there has been significant progress in human pose estimation, fueled by the widespread adoption of deep convolutional neural networks. However, despite these advancements, multi-person 2D pose estimation still remains highly challenging due to factors such as occlusion, noise, and non-rigid body movements. Currently, most multi-person pose estimation approaches handle joint localization and association separately. This study proposes a direct regression-based method to estimate the 2D human pose from a single image. The authors name this network YOLO-Rlepose. Compared to traditional methods, YOLO-Rlepose leverages Transformer models to better capture global dependencies between image feature blocks and preserves sufficient spatial information for keypoint detection through a multi-head self-attention mechanism. To further improve the accuracy of the YOLO-Rlepose model, this paper proposes the following enhancements. Firstly, this study introduces the C3 Module with Swin Transformer (C3STR). This module builds upon the C3 module in You Only Look Once (YOLO) by incorporating a Swin Transformer branch, enhancing the YOLO-Rlepose model’s ability to capture global information and rich contextual information. Next, a novel loss function named Rle-Oks loss is proposed. The loss function facilitates the training process by learning the distributional changes through Residual Log-likelihood Estimation. To assign different weights based on the importance of different keypoints in the human body, this study introduces a weight coefficient into the loss function. The experiments proved the efficiency of the proposed YOLO-Rlepose model. On the COCO dataset, the model outperforms the previous SOTA method by 2.11% in AP.

Bidirectional feature pyramid attention-based temporal convolutional network model for motor imagery electroencephalogram classification.

As an interactive method gaining popularity, brain-computer interfaces (BCIs) aim to facilitate communication between the brain and external devices. Among the various research topics in BCIs, the classification of motor imagery using electroencephalography (EEG) signals has the potential to greatly improve the quality of life for people with disabilities. This technology assists them in controlling computers or other devices like prosthetic limbs, wheelchairs, and drones. However, the current performance of EEG signal decoding is not sufficient for real-world applications based on Motor Imagery EEG (MI-EEG). To address this issue, this study proposes an attention-based bidirectional feature pyramid temporal convolutional network model for the classification task of MI-EEG. The model incorporates a multi-head self-attention mechanism to weigh significant features in the MI-EEG signals. It also utilizes a temporal convolution network (TCN) to separate high-level temporal features. The signals are enhanced using the sliding-window technique, and channel and time-domain information of the MI-EEG signals is extracted through convolution. Additionally, a bidirectional feature pyramid structure is employed to implement attention mechanisms across different scales and multiple frequency bands of the MI-EEG signals. The performance of our model is evaluated on the BCI Competition IV-2a dataset and the BCI Competition IV-2b dataset, and the results showed that our model outperformed the state-of-the-art baseline model, with an accuracy of 87.5 and 86.3% for the subject-dependent, respectively. In conclusion, the BFATCNet model offers a novel approach for EEG-based motor imagery classification in BCIs, effectively capturing relevant features through attention mechanisms and temporal convolutional networks. Its superior performance on the BCI Competition IV-2a and IV-2b datasets highlights its potential for real-world applications. However, its performance on other datasets may vary, necessitating further research on data augmentation techniques and integration with multiple modalities to enhance interpretability and generalization. Additionally, reducing computational complexity for real-time applications is an important area for future work.

A semi-supervised approach for the integration of multi-omics data based on transformer multi-head self-attention mechanism and graph convolutional networks.

Comprehensive analysis of multi-omics data is crucial for accurately formulating effective treatment plans for complex diseases. Supervised ensemble methods have gained popularity in recent years for multi-omics data analysis. However, existing research based on supervised learning algorithms often fails to fully harness the information from unlabeled nodes and overlooks the latent features within and among different omics, as well as the various associations among features. Here, we present a novel multi-omics integrative method MOSEGCN, based on the Transformer multi-head self-attention mechanism and Graph Convolutional Networks(GCN), withtheaimofenhancingtheaccuracyofcomplexdiseaseclassification. MOSEGCN first employs the Transformer multi-head self-attention mechanism and Similarity Network Fusion (SNF) to separately learn the inherent correlations of latent features within and among different omics, constructing a comprehensive view of diseases. Subsequently, it feeds the learned crucial information into a self-ensembling Graph Convolutional Network (SEGCN) built upon semi-supervised learning methods for training and testing, facilitating a better analysis and utilization of information from multi-omics data to achieve precise classification of disease subtypes. The experimental results show that MOSEGCN outperforms several state-of-the-art multi-omics integrative analysis approaches on three types of omics data: mRNA expression data, microRNA expression data, and DNA methylation data, with accuracy rates of 83.0% for Alzheimer's disease and 86.7% for breast cancer subtyping. Furthermore, MOSEGCN exhibits strong generalizability on the GBM dataset, enabling the identification of important biomarkers for related diseases. MOSEGCN explores the significant relationship information among different omics and within each omics' latent features, effectively leveraging labeled and unlabeled information to further enhance the accuracy of complex disease classification. It also provides a promising approach for identifying reliable biomarkers, paving the way for personalized medicine.

HRYNet: A Highly Robust YOLO Network for Complex Road Traffic Object Detection.

Object detection is a crucial component of the perception system in autonomous driving. However, the road scene presents a highly intricate environment where the visibility and characteristics of traffic targets are susceptible to attenuation and loss due to various complex road scenarios such as lighting conditions, weather conditions, time of day, background elements, and traffic density. Nevertheless, the current object detection network must exhibit more learning capabilities when detecting such targets. This also exacerbates the loss of features during the feature extraction and fusion process, significantly compromising the network's detection performance on traffic targets. This paper presents a novel methodology by which to overcome the concerns above, namely HRYNet. Firstly, a dual fusion gradual pyramid structure (DFGPN) is introduced, which employs a two-stage gradient fusion strategy to enhance the generation of more comprehensive multi-scale high-level semantic information, strengthen the interconnection between non-adjacent feature layers, and reduce the information gap that exists between them. HRYNet introduces an anti-interference feature extraction module, the residual multi-head self-attention mechanism (RMA). RMA enhances the target information by implementing a characteristic channel weighting policy, thereby reducing background interference and improving the attention capability of the network. Finally, the detection performance of HRYNet was evaluated by utilizing three datasets: the horizontally collected dataset BDD1000K, the UAV high-altitude dataset Visdrone, and a custom dataset. Experimental results demonstrate that HRYNet achieves a higher mAP_0.5 compared with YOLOv8s on the three datasets, with increases of 10.8%, 16.7%, and 5.5%, respectively. To optimize HRYNet for mobile devices, this study presents Lightweight HRYNet (LHRYNet), which effectively reduces the number of model parameters by 2 million. The results demonstrate that LHRYNet outperforms YOLOv8s in terms of mAP_0.5, with improvements of 6.7%, 10.9%, and 2.5% observed on the three datasets, respectively.